{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### AdaBoost的一般流程\n",
    "（1）收集数据：可以使用任意方法。  \n",
    "（2）准备数据：依赖于所使用的弱分类器类型，本章使用的是单层决策树，这种分类器可以处理任何数据类型。当然也可以使用任意分类器作为弱分类器，作为弱分类器，简单分类器的效果更好。  \n",
    "（3）分析数据：可以使用任意方法。  \n",
    "（4）训练算法：AdaBoost的大部分时间都用在训练上，分类器将多次在同一数据集上训练弱分类器。  \n",
    "（5）测试算法：计算分类的错误率。  \n",
    "（6）使用算法：同SVM一样，AdaBoost预测两个类别中的一个。如果想把它应用到多个类别的场合，那么就要像多类SVM中的做法一样对AdaBoost进行修改。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### AdaBoost（Adaptive boosting）的运行过程：\n",
    "- 训练数据中的每个样本，并赋予其一个权重，这些权重构成了向量$D$。  \n",
    "- 一开始，这些权重都初始化成相等值。  \n",
    "- 首先，在训练数据上训练出一个弱分类器并计算该分类器的错误率，然后在同一数据集上再次训练弱分类器。  \n",
    "- 在分类器的第二次训练当中，将会重新调整每个样本的权重，其中第一次分对的样本的权重将会降低，而第一次分错的样本的权重将会提高。  \n",
    "- 为了从所有弱分类器中得到最终的分类结果，AdaBoost为每个分类器都分配了一个权重值alpha，这些alpha值是基于每个弱分类器的错误率进行计算的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中，错误率$ε$的定义为：\n",
    "$$ε=\\frac{未正确分类的样本数目}{所有样本数目}$$\n",
    "而alpha的计算公式如下： \n",
    "$$α= \\frac{1}{2}ln(\\frac{1-ε}{ε})$$\n",
    "计算出alpha值之后，可以对权重向量$D$进行更新，以使那些正确分类的样本的权重降低而错分样本的权重升高。$D$的计算方法如下： \n",
    "如果某个样本被正确分类，那么该样本的权重更改为： \n",
    "$$D_i^{(t+1)}=\\frac{D_i^{(t)}e^{-α}}{Sum(D)}$$\n",
    "而如果某个样本被错分，那么该样本的权重更改为;  \n",
    "$$D_i^{(t+1)}=\\frac{D_i^{(t)}e^{α}}{Sum(D)}$$\n",
    "在计算出$D$之后，AdaBoost又开始下一轮迭代。AdaBoost算法会不断重复训练和调整权重的过程，直到错误率为0或者弱分类器的数目达到用户的指定值为止。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\"\"\"\n",
    "函数说明:加载数据\n",
    "\n",
    "Parameters:\n",
    "    无\n",
    "Returns:\n",
    "    datMat - 数据列表\n",
    "    classLabels - 标签列表\n",
    "\"\"\"\n",
    "def loadSimpData():\n",
    "    datMat = np.matrix([[1.,2.1],\n",
    "                        [2.,1.1],\n",
    "                        [1.3,1.],\n",
    "                        [1.,1.],\n",
    "                        [2.,1.]])\n",
    "    classLabels = [1.0,1.0,-1.0,-1.0,1.0]\n",
    "    return datMat,classLabels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "函数说明:单层决策树分类函数\n",
    "\n",
    "Parameters:\n",
    "    dataMatrix - 数据矩阵\n",
    "    dimen - 第dimen列，即第dimen个特征\n",
    "    threshVal - 阈值\n",
    "    threshIneq - 标志，lt--less than；gt--great than\n",
    "Returns:\n",
    "    retArray - 分类结果\n",
    "\"\"\"\n",
    "def stumpClassify(dataMatrix,dimen,threshVal,threshIneq):\n",
    "    retArray = np.ones((dataMatrix.shape[0],1))     #将分类结果初始化为1\n",
    "    if threshIneq == 'lt':\n",
    "        retArray[dataMatrix[:,dimen] <= threshVal] = -1.0   #如果小于阈值，则分类结果为-1\n",
    "    else:\n",
    "        retArray[dataMatrix[:,dimen] > threshVal] = -1.0    #如果大于阈值，则分类结果为-1\n",
    "    return retArray"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "函数说明:找到数据集上最佳的单层决策树\n",
    "\n",
    "Parameters:\n",
    "    dataArr - 数据列表\n",
    "    classLabels - 标签列表\n",
    "    D - 样本权重\n",
    "Returns:\n",
    "    bestStump - 最佳单层决策树信息\n",
    "    minError - 最小错误率\n",
    "    bestClassEst - 最佳分类结果\n",
    "\"\"\"\n",
    "def buildStump(dataArr, classLabels,D):\n",
    "    dataMatrix = np.mat(dataArr); labelMat = np.mat(classLabels).T  #将训练数据和标签列表转换为numpy矩阵\n",
    "    m,n = np.shape(dataMatrix)  #返回训练数据的大小\n",
    "    numSteps = 10.0; bestStump = {}; bestClassEst = np.mat(np.zeros((m,1))) #初始化\n",
    "    minError = np.inf   #将最小错误率设置为无穷大\n",
    "    for i in range(n):  #遍历所有特征\n",
    "        rangeMin = dataMatrix[:,i].min(); rangeMax = dataMatrix[:,i].max()  #找到特征中最小和最大的值\n",
    "        stepSize = (rangeMax-rangeMin)/numSteps #计算步长\n",
    "        for j in range(-1,int(numSteps)+1):\n",
    "            for inequal in ['lt','gt']:     #遍历大于和小于的情况\n",
    "                threshVal = (rangeMin+float(j)*stepSize)    #计算阈值\n",
    "                predictedVals = stumpClassify(dataMatrix,i,threshVal,inequal)   #计算分类结果\n",
    "                errArr = np.mat(np.ones((m,1)))     #初始化误差矩阵\n",
    "                errArr[predictedVals == labelMat] = 0   #若预测正确，赋值为0\n",
    "                weightedError = D.T*errArr  #计算加权误差率\n",
    "                #print(\"split: dim  %d, thresh  %.2f, thresh ineqal: %s, the weighted error is %.3f\" %\\\n",
    "                 #     (i, threshVal,inequal,weightedError))\n",
    "                if weightedError < minError:    #找到误差最小的分类方式\n",
    "                    minError = weightedError\n",
    "                    bestClassEst = predictedVals.copy()\n",
    "                    bestStump['dim'] = i    #将最佳单层决策树的信息存至字典\n",
    "                    bestStump['thresh'] = threshVal\n",
    "                    bestStump['ineq'] = inequal\n",
    "    return bestStump,minError,bestClassEst"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "函数说明:基于单层决策树的AdaBoost训练过程\n",
    "\n",
    "Parameters:\n",
    "    dataArr - 数据集\n",
    "    classLabels - 类别标签\n",
    "    numIt - 迭代次数\n",
    "Returns:\n",
    "    weakClassArr - 各层决策树信息\n",
    "\"\"\"\n",
    "def adaBoostTrainDS(dataArr, classLabels, numIt = 40):\n",
    "    weakClassArr = []\n",
    "    m = np.shape(dataArr)[0]\n",
    "    D = np.mat(np.ones((m,1))/m)    #初始化权重\n",
    "    aggClassEst = np.mat(np.zeros((m,1)))   #记录每个数据点的类别估计累计值\n",
    "    for i in range(numIt):\n",
    "        bestStump, error, classEst = buildStump(dataArr,classLabels,D)  #构建单层决策树\n",
    "        #print(\"D: \", D.T)\n",
    "        alpha = float(0.5*np.log((1.0-error)/max(error,1e-16))) #根据公式计算弱学习算法权重\n",
    "        bestStump['alpha'] = alpha  #存储弱学习算法权重\n",
    "        weakClassArr.append(bestStump)    #存储单层决策树\n",
    "        #print(\"classEst: \", classEst.T)\n",
    "        expon = np.multiply(-1*alpha*np.mat(classLabels).T,classEst)    #根据公式计算e的指数项\n",
    "        D = np.multiply(D,np.exp(expon))\n",
    "        D = D/D.sum()   #根据样本权重公式，更新权重\n",
    "        aggClassEst += alpha*classEst   #将各数据点的类别估计值累加\n",
    "       # print(\"aggClassEst: \", aggClassEst.T)\n",
    "        aggErrors = np.multiply(np.sign(aggClassEst)!= np.mat(classLabels).T, np.ones((m,1)))   #计算误差\n",
    "        errorRate = aggErrors.sum()/m\n",
    "        print(\"total error:\", errorRate,\"\\n\")\n",
    "        if errorRate == 0.0: break\n",
    "    return weakClassArr,aggClassEst"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "函数说明:AdaBoost分类函数\n",
    "\n",
    "Parameters:\n",
    "    datToClass - 待分类数据集\n",
    "    classifierArr - 训练好的多个弱分类器数组\n",
    "Returns:\n",
    "    分类结果\n",
    "\"\"\"\n",
    "def adaClassify(datToClass, classifierArr):\n",
    "    dataMatrix = np.mat(datToClass) #将待分类数据集转换为numpy矩阵\n",
    "    m = np.shape(dataMatrix)[0] #求得待分类数据的样本数\n",
    "    aggClassEst = np.mat(np.zeros((m,1))) #初始化分类结果\n",
    "    for i in range(len(classifierArr)): #遍历所有的分类器，进行分类\n",
    "        classEst = stumpClassify(dataMatrix,classifierArr[i]['dim'],classifierArr[i]['thresh'],\\\n",
    "                                 classifierArr[i]['ineq'])\n",
    "        aggClassEst += classifierArr[i]['alpha']*classEst   #给样本加上权重\n",
    "        #print(aggClassEst)\n",
    "    return np.sign(aggClassEst) #返回分类结果"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 示例：在一个难数据集上应用AdaBoost\n",
    "（1）收集数据：提供的文本文件。  \n",
    "（2）准备数据：确保类别标签是+1和-1而非1和0。  \n",
    "（3）分析数据：手工检查数据。  \n",
    "（4）训练算法：在数据上，利用adaBoostTrainDS()函数训练出一系列的分类器。  \n",
    "（5）测试算法：我们拥有两个数据集。在不采用随机抽样的方法下，我们就会对AdaBoost和Logistic回归的结果进行完全对等的比较。  \n",
    "（6）使用算法：观察该例子上的错误率。不过，也可以构建一个Web网站，让驯马师输入马的症状然后预测马是否会死去。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "函数说明:自适应数据加载函数\n",
    "\n",
    "Parameters:\n",
    "    fileName - 文件名\n",
    "Returns:\n",
    "    dataMat- 数据集\n",
    "    labelMat- 标签列表\n",
    "\"\"\"\n",
    "def loadDataSet(fileName):\n",
    "    numFeat = len(open(fileName).readline().split('\\t'))    #获取特征数目\n",
    "    dataMat = []; labelMat = []\n",
    "    fr = open(fileName) #打开文件\n",
    "    for line in fr.readlines(): #逐行读取\n",
    "        lineArr = []\n",
    "        curLine = line.strip().split('\\t')  #去掉回车，放入列表\n",
    "        for i in range(numFeat-1):\n",
    "            lineArr.append(float(curLine[i]))\n",
    "        dataMat.append(lineArr) #添加数据\n",
    "        labelMat.append(float(curLine[-1])) #添加标签\n",
    "    return dataMat,labelMat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "total error: 0.284280936455 \n",
      "\n",
      "total error: 0.284280936455 \n",
      "\n",
      "total error: 0.247491638796 \n",
      "\n",
      "total error: 0.247491638796 \n",
      "\n",
      "total error: 0.254180602007 \n",
      "\n",
      "total error: 0.240802675585 \n",
      "\n",
      "total error: 0.240802675585 \n",
      "\n",
      "total error: 0.220735785953 \n",
      "\n",
      "total error: 0.247491638796 \n",
      "\n",
      "total error: 0.230769230769 \n",
      "\n"
     ]
    }
   ],
   "source": [
    "dataMat,labelMat = loadDataSet('data/horseColicTraining2.txt')\n",
    "classifierArray,aggClassEst = adaBoostTrainDS(dataMat,labelMat,10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.23076923076923078"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "testeMat,testLabelMat = loadDataSet('data/horseColicTraining2.txt')\n",
    "prediction10 = adaClassify(testeMat,classifierArray)\n",
    "#求得预测错误率\n",
    "m,n = np.shape(testeMat)\n",
    "errArr = np.mat(np.ones((m,1)))\n",
    "errArr = np.multiply((prediction10!=np.mat(testLabelMat).T),errArr)\n",
    "errRate = errArr.sum()/m\n",
    "errRate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 其他分类性能度量指标：正确率、召回率及ROC曲线\n",
    "- 混淆矩阵中有真正例（TP）、伪正例（FP）、伪反例（FN）、真反例（TN）\n",
    "- 正确率为TP/(TP+FP)，给出的是预测为正例的样本中的真正正例的比例。\n",
    "- 召回率为TP/(TP+FN)，给出的是预测为正例的真实正例占所有真实正例的比例。\n",
    "- ROC曲线（ROC curve），ROC代表接收者操作特征（receiver operating characteristic）。\n",
    "- ROC曲线下的面积AUC(Area Under the Curve)给出的是分类器的平均性能值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "函数说明:ROC曲线的绘制及AUC计算函数\n",
    "\n",
    "Parameters:\n",
    "    predStrengths - 分类器的预测强度\n",
    "    classLabels - 标签列表\n",
    "Returns:\n",
    "    无\n",
    "\"\"\"\n",
    "def plotROC(predStrengths, classLabels):\n",
    "    import matplotlib.pyplot as plt\n",
    "    cur = (1.0,1.0)     #绘制光标的位置\n",
    "    ySum = 0.0          #用于计算AUC\n",
    "    numPosClas = sum(np.array(classLabels)==1.0)    #统计正类的数量\n",
    "    yStep = 1/float(numPosClas)                     #确定y轴的步长:TP/(TP+FN)\n",
    "    xStep = 1/float(len(classLabels)-numPosClas)    #确定x轴的步长：FP/(FP+TN)\n",
    "    sortedIndicies = predStrengths.argsort()        #得到排序索引\n",
    "    fig = plt.figure()                              #构建画笔\n",
    "    fig.clf()\n",
    "    ax = plt.subplot(111)\n",
    "    for index in sortedIndicies.tolist()[0]:        #在所有排序值上进行循环\n",
    "        if classLabels[index] == 1.0:\n",
    "            delX = 0; delY = yStep                  #每得到一个正类，就沿着y轴方向下降一个步长，不断降低真阳率\n",
    "        else:\n",
    "            delX =xStep; delY=0                     #其他类就沿着x轴方向倒退一个步长\n",
    "            ySum += cur[1]                          #计算面积是小矩形的宽度是xStep，高度累加\n",
    "        ax.plot([cur[0],cur[0]-delX],[cur[1],cur[1]-delY],c='b')\n",
    "        cur = (cur[0]-delX,cur[1]-delY)\n",
    "    ax.plot([0,1],[0,1],'b--')\n",
    "    plt.xlabel('False Positive Rate'); plt.ylabel('True Positive Rate')\n",
    "    plt.title('ROC curve for AdaBoost Horse Colic Detection System')\n",
    "    ax.axis([0,1,0,1])\n",
    "    plt.show()\n",
    "    print('the Area Under the Curve is: ', ySum*xStep)      #计算AUC\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xu8VPP6wPHPU0oo5dI56I6cJFS2yjV3lUsRikRCrsdxPTi8jsuPc3Dwc5zjFjrhx3G/hMhxqYRSiVDSBbVz2yhHVLo8vz+e79irMTN79t4ze83Mft6v137tWTNr1jxrzcx6Zn2/3/UsUVWcc865dBrEHYBzzrnC5onCOedcRp4onHPOZeSJwjnnXEaeKJxzzmXkicI551xGnihKjJh/icgSEXk7phhGi8g1cby2y0xErhSR/wu324rIMhFpGHdcuSQiQ0TkpbjjKCUlkShE5FMRWR4+9F+GHVXTpHl2F5FXReQHEfleRJ4Vkc5J82wsIreIyMKwrHlhevO6XaNa2RM4EGitqj1ytVAR2Shsk7G5WmZY7ngRWRGW/b2ITBSRHXP5Giles8pEJiIqItsm3ffLTjZOItI4xDJXRH4Mn/9RItK+OstR1YWq2lRV11Tz9YeJyJrwni0TkU/Cj5PtqrGM8SJySnVeN81y2of3ar3Efar6oKoeVNtlp3m9P4X1XSYi5SLySC2Xt4+IlOcqvnwpiUQRHKaqTYGuQDfg0sQDIrIb8BLwDLAV0AF4D3hDRLYO8zQGXgF2APoAGwO7A98COdvhJot+wHOkHfCpqv6Y41iOAlYCB4nIljUNLo2zw3u3GTAeeCDHy49VOMrL5XftceBw4DigObAzMB3YP4evUZW3wnvWHDgAWA5MF5EudRhDnRKRE4GhwAFh3cuwfUbpU9Wi/wM+xd68xPQNwPOR6deB21M87wXg/nD7FOAroGk1XncH4D/Ad+G5fwr3jwauicy3D1CeFO/FwExs53s58HjSsv8O3BpuNwfuBb4AFgPXAA1TxHMysAJYAywDrgr3nwrMC3GOAbaKPEeBs4C5wCcZ1vVV4FrgHeDCpMe6hft/AB4BHk6sP7AJ8BxQASwJt1tHnjseOCUy3Rn4OTK9PnAL8Hn4uwVYP/J4ynUDBPhf4Gvg+7CtuwAjgFXAz2EbPZtmfRXYNum+K4H/i0zvDkwNy58K7J60XtcCb2A70W2BYcCCsJ0+AYZE5h8OzA7baBzQLk1ciZ1ymwzv1VZhW3wXts2pqdYBaB/Wc70wvSnwr7CdlwBPp1n+MGBSivufI/I5BnoBbwJLsR9m+4T7r8U+oyvCe/DPcH8nKr9Pc4BjIsvaALgJ+Cxs70nhvoVhHZaFv92S48viffqf8D79gP2g3DzNev8TuCXNY0cD05PuuyCxDYF+wKzwGouBC4GNwnu5NhL/VtgP+EuA+dgP1UeBTZPes5OAReF9Oh3YFfuML01sz1z+xb6Tz8lKRBIF0Bp4H/h7mN4wfCj3TfG8k4Avwu2Hgfuq8ZrNsB33BUCTMN0zPDaaqhPFu0Cb8GFvB/wEbBwebxiW3StMPw3cFT5YvwHeBk7L5ksM7Ad8A3THdrr/ACZGHlfsy7kpsEGaZbYNH+bOYX1nRh5rjH15zwMaYUceq6hMFJsBA8P70Ax4jMgOiEiiCMu6Nim+q4HJYb1bYjue/6lq3YCDsV/ZLbCksT2wZar3J806Z0wUYXstwX5hrgccG6Y3i6zXQuzHxHpYsv8v8Lvw+JbADuH2AGyHvn2Y93LgzTRxXQdMqCL2CcDt2OeyK5ak90+xDu1ZN1E8jyX6TcJ72Tubz1jk/uHAV+F2K2wn1w/b8R0Yplsmv+9heiNsx3dS2Abdw3ub2Ea3hee0wr4fu4f3fJ11SI4vy/dpPrAd9l0cD1yXZr2Px5LYRdjRRMPIY+uHx7aP3DcDGBhufwHsFW5vAnRPtW8I952LfeZbh+XeBfw76T27M7y/B2EJ92nsO9IK+3GU8r2r6V/sO/mcrITteJdh2Vqxw8EW4bHW4b5OKZ7XB1gVbv8n3QckzWseC8xI89hoqk4Uw5OeMwk4Idw+EJgfbv8WO+rYIOm1X0vz2r98ScL0vcANkemm2I68fZhWYL8q1vVy4N1weyss8XYL03tjv0AlMv+bpNkRYzuuJZHp8ViSXIr9yv+esFMLj88H+kWmD8aa1jKuG5ZEPsZ+1TbI9P6kiVOxHfvSyN8KKneyQ4G3k57zFjAssl5XRx7bKCxjIEkJGTuyPTky3SBsk3Yp4robeDhD3G3C+9Msct9fgdHh9pWkSBRY4loLbJLFZ3+dz1ia79PFwANJj48DToxsn2iiGAS8njT/XcAVYXssB3ZO8Zq/rEOq+LJ8ny6PPHYm8GKGdR8CvAz8iCW+SyKP3QFcG27vgCWk9cP0QuA0wo/ByHP24deJYjbrfge2xD7X60XWt1Xk8W+BQZHpJ4Bzq3ofq/NXSn0UA1S1GbbhOwGJDugl2BcgVbv6ltivFrCNXZ229zbYTqymFiVNP4QlALC254fC7XbYr7svRGSpiCzFvkC/yfJ1tsJ+8QOgqsuwdW2VIZZkJwAPhud/jv1iPTGy/MUaPqHBL68nIhuKyF0i8pmI/BeYCLRIGmlzjqq2wH4hHQo8LiI7pYo/3N6qqnVT1VexpoLbgK9EZKSIbFzFeibrrqotEn/Yr/mE5LgSsaXcrmp9RoOwZoIvROR5EekUHm4H/D3y/n6HHQVFl5VQ1ed0K+A7Vf0hQ1yptAnPW1LFfJm0wmIHW6ejE+sU1mtP0sfeDuiZNP8QYAvsu9yEmn3fsnmfvozc/gn7wZGSWkf5AdiR6unA1SJycHj4PuA4EREsQT2qqivDYwOxo6vPRGRC6DdNpx3wVGQ7zMaS/28j83wVub08xXTadaiJUkoUAKjqBOwX441h+kfsF8TRKWY/hsrOqJeBg0VkoyxfahGwTZrHfsSaWhK2SBVq0vRjwD4i0ho4gspEsQg7otg8stPaWFV3yDLOz7EPHmCjl7DmoMUZYvmFiOwOdAQuDSPKvgR6AseGzu8vgFbhy5HQNnL7AuB3WLPcxtgRCNiOcB2qulZVX8eaYRKjVtaJPyz782zWTVVvVdVdsF9322FNBhnXtxqS40rElna7quo4VT0Q21l+hB0dgL3Hp0WTkqpuoKpvpnjdl4Ee4XOSLq5NRaRZhrhSWRSe16KK+TI5AusPTCzvgaR12khVE8k2+T1YhDWpRedvqqpnYD/mVpD6+1bVe5nN+1RtqrpKVR+jsu8LVZ2MHRXvhf3YeyAy/1RV7Y/9wHsa63dIF/8ioG/StmiiqrWKuTZKLlEEtwAHikjXMH0JcKKInCMizURkkzA8cjfgqjDPA9gb9ISIdBKRBiKyWRgO1y/FazwHbCEi54rI+mG5PcNj7wL9RGRTEdkCa3PMSFUrsMPgf2GdyrPD/V9gHWw3heG7DURkGxHpneW2eAg4SUS6isj6wF+AKar6aZbPPxFrluuMNRt1xb4YGwJ9sSS8GjhHRNYTkSNZd5RYM+wXzlIR2RRrSkgr/NLqDHwY7vo3cLmItAzDlP8MJIaopl03EdlVRHqKSCMscSc6+cF+fW2d5fqnMxbYTkSOC+s9KMT9XJr1+q2IHB6S2UqsqTQRz51YIt4hzNtcRFL9sEFVX8bej6dEZJfw2s1E5HQRGa6qi7Cmv7+KSJNwZHYy4YgwnfA5ewG4PXw/GonI3pmeE2JtKCIdROQf2NF84vv0f8BhInJwmKdJGAqaSHDJ78Fz2PYcGl67UXgPt1fVtcAo4GYR2Sosb7fwnldgLQbp3s9qvU9VrOswETkkbO8GItIX+xEyJTLb/diR7GpVnRSe11js3I7mqroKa9KMfhY3E5HmkWXcCVwrIu3C81uKSP/qxptTuWzHiuuPpFFPWtle+ERkek9sR7wMe6OeB7okPac5lmQWhfnmAzcTOr5SvG4X7IhkCXb4ekm4vwnWKfhf7BfHefy6j+KAFMsbiv3CuChFXHcA5Vgb/gxgcJqYhpHUfowdIs/HmgWSRx0pSZ22kceahHU7LMVjtxNGuGAdezOoHPX0CJWd2VtFtvvHWDvtL23K4bHE6Jdl2NHEeUkx3IoduXwRbjepat2woaIzwzK/wXaUTcNjHbFkvpT0I3t+tV349ainPbEO8+/D/z0jj41n3Tb4LbEmu+/D644HOie99++Hz8wiYFSGz3tjbIc8D0uCnwH3AG3D463DtvgubJvTU60DqUc93YftvJYAT2b4jCVG1iVe/z4iHblhvp5hnb/DdujPR2LcLXwellA5uu93YZ4KrIntVaBreGwD7Lu5OGzDiYS+HmzAQ0XYrr34dT9ddd6ndZ6btD5HYqOjloT36X1CX0dknsTAj6uS3q8XI8+bmhTDqLC+S6kc9XQ+NvLrh/Ae/iXVexbuKyeMKAvT/0ek3yUXfxIW7JxzrpZEZANs1FF3VZ0bdzy5UqpNT845F4czgKmllCQgj4lCrKTA1yLyQZrHRURuFSuTMVNEuucrFuecyzcR+RT4AzaAo6Tk84hiNDauOp2+WFtxR+xs2TvyGItzzuWVqrZX1XaqOiPuWHItb4lCVSdSOaY6lf5Y+QxVG1bWQnJfQ8g551wt5bogXXW0Yt0TvcrDfV8kzygiI7CjDjbaaKNdOnXqlDyLc87FauZMWLsWNtgAVobT7NZfv/J2QvS+bG5X53mp5gVYvRpUp3+jqi1rsm5xJopfnXBFmpNnVHUkMBKgrKxMp02bls+4nHMlrFs3qKiAbbeFefPsvujthHSPp5t3ww2haVMoL4Ci4YnBrCJwxx3w9ddw5ZWSfIZ61uJMFOVY2YCE1lSeceucc79S1U4+mx3+rFmVt3OpaVNoWaPf67m1eDGccQYMGgRDhthtgCuvrPky40wUY4CzReRh7MSc79XODnXOlahUO/qEbHb+FRWV99VUy5b2N358zZdRiFThnnvgwgth1So45JDcLTtviUJE/o2d0r+52BWcrsCK26Gqd2Kn1vfDzi79CSsv7JwrAjXd4df213yp7uRra/58OPVUeO012HdfuPtu2CZdJboayFuiUNVjq3hcsQvmOOcKQHV2/jXd4fuOPj/efx+mT4eRI+GUU6xvIpfibHpyzhWQigpYtiy7eX2HH78PPoB33oETToABA2DBAthss/y8licK5+qx6FHEsmXWIes7/8L288/wl7/Y329/C8ccA02a5C9JgNd6cq5emzu3soO4UEbtuPSmTIHu3eGqq2xU04wZliTyzY8onKtnokcRq1ZBo0Z+FFEMFi+Gvfayo4jnnsvtqKaqeKJwroSl6qCODjFN9DW4wvXxx7DddtCqFTzyCOy/P2xc3Yv61pI3PTlXwlJ1ULdsCZ0721FEebk1X7jCs3QpjBgBnTrBxIl23xFH1H2SAD+icK7keQd18Rkzxs6o/vJLuOgi2HXXeOPxROFcCVu6NO4IXHWdcgrcey/suCM88wyUlcUdkScK54pOdU6MS3RWu8IWLeJXVgbt2sHFF0PjxvHGleCJwrkiM3euJYBszor2zurCt2gRnH46DB4MQ4fa7ULjicK5IuBDWkvP2rVw11125LBmjXVUFypPFM4VgejoJT9KKH5z51pfxMSJcMABVqOpQ4e4o0rPE4VzBcrLa5SuWbPsinijRsGwYbkv4pdrniicq6V8XTEtWqHVy2sUv/feg3ffhRNPhP79rYjfJpvEHVV2PFE4V0vV6VyuDq/QWhpWroRrroHrroMtt7QaTU2aFE+SAE8Urp5KHAUkZPNrP90vf+9cdum89RacfDLMnm3lwG++uW6K+OWaJwpXLyU6h5s2rf2yvHPZpbJ4MfTuDVtsAWPHQt++cUdUc54oXMmpqs8g2jlcXh5fnK40zZ4N229vRfwefdSK+DVrFndUteNFAV3R6dYNWreGffax/8m3Z81at1kpFe8cdrm2ZAkMH24FF19/3e4bMKD4kwT4EYUrQlV1HnsnsKtrTz0FZ55pP1AuvTT+In655onCFQU/M9kVquHD4V//gq5d4fnn7Qp0pcYThStoiQThF9txhSRaxK9XL+jYES68sHQLMHqicAUtMTrJm5NcofjsMzjtNDjuOBvyOmJE3BHln3dmu4IT7ayOjk7yK7G5OK1dC7fdBl26wKRJ1gRaX/gRhSsI0T4IL13hCs2cOVbEb9IkOOggq/ravn3cUdUdTxSuIERHMnkzkys0c+bAhx/C6NHW3FToRfxyzROFKxg+kskVkhkzrIjfSSfB4YdbEb8WLeKOKh7eR+EKQosW9fdL6ArLihXwpz/ZuRBXXmnTUL8/n35E4epEVdd5zlXdJedq4403rIjfnDl2JHHTTcVZxC/XPFG4vEnXQZ2Kd1q7uC1eDPvuazWaxo2zTmtnPFG4nEqXHLyD2hWqWbOsPlOrVvDEE5Ys/Oh2XZ4oXJWqcwU3Tw6uWHz3HZx/Ptx3H0yYAHvvDYcdFndUhckThUupOs1GUZ4cXDF44gk46yz49lu47DLo0SPuiAqbJwqXUqJ0BvjO35WWYcPsKKJ7d3jxRSvm5zLzROHSatrUk4MrDdEifrvvbhcWuuACWM/3gFnJ63kUItJHROaIyDwRuSTF421F5DURmSEiM0WkXz7jcamluhBQRQUsXRp3ZM7V3ief2Aim+++36REj4OKLPUlUR94ShYg0BG4D+gKdgWNFpHPSbJcDj6pqN2AwcHu+4nHpzZ376yvCtWxppZOdK1Zr1sCtt1oRv8mTK48qXPXlM6f2AOap6gIAEXkY6A/MisyjwMbhdnPg8zzG4zLw8hmulMyebSfOvfUW9O0Ld94JbdvGHVXxymeiaAUsikyXAz2T5rkSeElEfg9sBByQakEiMgIYAdDW3+2cq8+lCVxpmjfPzq5+4AEYMqT+FfHLtXz2UaR6a5IP/o4FRqtqa6Af8ICI/ComVR2pqmWqWtbST991zqUwfTqMGmW3DzvM+iaOP96TRC7k84iiHGgTmW7Nr5uWTgb6AKjqWyLSBNgc+DqPcdU7iXMiEpJPnKuoKN1LOLrSt3w5XHUV3HgjtGljV55r0gQ23rjq57rs5POIYirQUUQ6iEhjrLN6TNI8C4H9AURke6AJkNSt6morVWd1lHdcu2I1cSLsvDNcf72dHzFjhhfxy4e8HVGo6moRORsYBzQERqnqhyJyNTBNVccAFwB3i8h5WLPUMFUfm5AL0TOrV62yI4by8rijci53Fi+G/fe3o4iXX7bbLj/yOpJYVccCY5Pu+3Pk9ixgj3zGUF+lumKcc6Xg/fdhxx2tiN9TT1kRv402ijuq0uYXLipRLVpUlt0oL7dDcueK2TffwNChsNNO1uQEcOihniTqgp+b6JwraKrw2GNw9tmwZAlccQX0TB5o7/LKE0URqqrst49kcqXkxBPtfIiyMnjlFWt2cnXLE0URivY/pOP9Eq6YRYv49e5tzU3nnuv1meLim71IeckNV6oWLIBTT7WT5U46yUpxuHh5Z3YRatHCy2640rNmDdxyizUtTZ0KDXzvVDD8iMI5F7tZs2D4cJgyBQ45xIr4tW4dd1QuwRNFEfLrRLhS88knMH8+PPQQDB7s9ZkKjScK51wspk6Fd9+1/ohDDrG+iWbN4o7KpeKtgM65OvXTT3DhhdCrF/z1r7Bihd3vSaJweaJwztWZ8eNtqOtNN9mRhBfxKw7e9FSEfMSTK0bl5XDggdCuHbz6qtVocsXBjyicc3n13nv2v3VreOYZmDnTk0Sx8URRhJYu9ZFPrvBVVNhFhLp2hQkT7L5+/WDDDeONy1WfNz0ViGzqNyVuJ64v4VwhUoWHH4ZzzoHvv7erz+22W9xRudrIKlGEK9S1VdV5eY6n3kkkiMQV6DLVb0rwOk6ukA0dCg8+aBVe770Xdtgh7ohcbVWZKETkEOBmoDHQQUS6Aleo6hH5Dq4+qKiAZcsqd/5ev8kVo7Vr7SQ5Eet/2GUXO6Jo2DDuyFwuZHNEcTXQE3gNQFXfFZEsfve6bDVt6pcpdcVr3jwb6jp0qJXh8CJ+pSebzuxVqprcderXtc4R75h2xWr1arjxRiviN2MGNG4cd0QuX7I5opgtIscADUSkA/AHYHJ+w3LOFbIPPrAS4NOmQf/+cPvtsNVWcUfl8iWbI4qzgV2AtcCTwAosWTjn6qmFC+Gzz2x001NPeZIoddkcURysqhcDFyfuEJEjsaThaiA6FNaHurpiMWWKnTw3YoSdD7FggfWvudKXzRHF5SnuuyzXgdQniZFOYCOdOnaMNx7nMvnxRzj/fDsX4oYbYOVKu9+TRP2R9ohCRA4G+gCtROTmyEMbY81QLo3EEUNC8olzy5bZl8yHwrpC9+qrNqJpwQI44wy47jpYf/24o3J1LVPT09fAB1ifxIeR+38ALslnUMUo2pw0a5bdl+6kuKZN/YQ5V/jKy+Hgg6FDByvBsffecUfk4pI2UajqDGCGiDyoqivqMKaClq7URiI5bLtt5clzM2bEF6dzNTVjhn3OW7eGZ5+F3r1hgw3ijsrFKZs+ilYi8rCIzBSRjxN/eY+sQM2du26zUkLLltC5szUnlZd7knDF56uvYNAg6N69sohfnz6eJFx2o55GA9cANwJ9gZOox30UiWtBeP+CKxWqVpvpD3+w/rNrroHdd487KldIsjmi2FBVxwGo6nxVvRzwavLOlYjjjrPyG7/7nV3D+rLLfMi2W1c2RxQrRUSA+SJyOrAY+E1+wyoMqfojKir8S+SKX7SI30EH2dDXs87yIn4utWyOKM4DmgLnAHsApwLD8xlUoUjVH+HnPbhi9/HHVuF11CibPukkr/TqMqvyiEJVp4SbPwBDAUSkdT6DKiSNGnl/hCsNq1fDzTfDFVdAkybeSe2yl/GIQkR2FZEBIrJ5mN5BRO7HiwI6V1RmzoReveDii6FvXxvOfdxxcUflikXaRCEifwUeBIYAL4rIZdg1Kd4Dtqub8OLVokXlKCfnill5OSxaBI89Bk88AVtuGXdErphkanrqD+ysqstFZFPg8zA9J9uFi0gf4O9AQ+AeVb0uxTzHAFdi17h4T1X9d45zOfDmm3YkcfrplUX8Ntoo7qhcMcrU9LRCVZcDqOp3wEfVTBINgduwcy86A8eKSOekeToClwJ7qOoOwLnVjD+v/KJCrhgtW2bnROy5J9x0U2URP08SrqYyHVFsLSKJUuICtI9Mo6pHVrHsHsA8VV0AICIPY0cpsyLznArcpqpLwjK/rmb8OeclwF0xe+klKwO+cKENd/3LX7yIn6u9TIliYNL0P6u57FbAosh0OXbt7ajtAETkDax56kpVfTF5QSIyAhgB0LZt22qGUT1z51qCiNZscq4YLFoEhxwC22wDEyfaEYVzuZCpKOArtVy2pFpsitfvCOwDtAZeF5EuydfoVtWRwEiAsrKyvF6v20t0uGIzfTrssgu0aQNjx8Jee9nwV+dyJZsT7mqqHGgTmW6NdYgnz/OMqq5S1U+AOVjicM5V4csv4eijoayssojfgQd6knC5l89EMRXoKCIdRKQxMBgYkzTP04S6UeFcje2ABXmMqUrege0KnSrcd59VK372WeuH8CJ+Lp+yqfUEgIisr6ors51fVVeLyNnAOKz/YZSqfigiVwPTVHVMeOwgEZkFrAEuUtVvq7cKztUvgwfDo4/CHnvAPfdAp05xR+RKnahmbvIXkR7AvUBzVW0rIjsDp6jq7+siwGRlZWU6bdq0nC4zOtLprbdspFPimtbOFYJoEb/77oMffoAzz4QG+WwTcCVFRKarallNnpvNx+xW4FDgWwBVfY8SKzMeLf7nRf9cofnoI7sM6b332vSJJ8LZZ3uScHUnm6anBqr6mVUa/8WaPMUTCx/p5ArRqlXwt7/BVVfZyXJNm8YdkauvskkUi0Lzk4azrX8P1NtLoTpXF95918p/v/suHHUU/OMfsMUWcUfl6qtsEsUZWPNTW+Ar4OVwX8nwUU6u0Hz5pf098QQcWVUNBOfyLJtEsVpVB+c9EufquUmTrIjfmWdCnz4wfz5suGHcUTmXXaKYKiJzgEeAJ1X1hzzHlHOpLmma4DWdXNx++AEuvRRuu80GUpx8stVn8iThCkWV4yZUdRvgGmAX4H0ReVpEiuoIo6Ii83BXH+nk4jJuHHTpArffbhVf33nHi/i5wpPVCXeq+ibwpohcCdyCXdDo4TzGlXNNm/qoJldYFi2CQw+1o9pJk/zsale4qkwUItIUKw8+GNgeeAYo+I90tLmposKbllxhUIWpU6FHDyvi98ILVuXV6zO5QpbNKTsfAL2AG1R1W1W9QFWn5DmuWvOT6Fyh+eILGDgQevasLOJ3wAGeJFzhy6bpaWtVXZv3SPKgUSNvbnLxU4XRo+H882HFCrj+eqvT5FyxSJsoROQmVb0AeEJEflUQKosr3DnngGOOgccft+tE3HMPbLdd3BE5Vz2ZjigeCf+re2W7gpAoy+FcHNassQJ+DRrAYYfBfvvBaad5fSZXnNJ+bFX17XBze1V9JfqHdWo751KYPduOHhJF/E44Ac44w5OEK17ZfHSHp7jv5FwHkmt+ASJX11atgmuuga5dYc4caN487oicy41MfRSDsCGxHUTkychDzQDfBTsXMWMGDBtmJTgGDYJbb4Xf/CbuqJzLjUx9FG9j16BoDdwWuf8HYEY+g3Ku2Hz1FXzzDTz9NPTvH3c0zuVW2kShqp8An2DVYouOd2a7fJs4Ed5/H846y4r4zZsHG2wQd1TO5V7aPgoRmRD+LxGR7yJ/S0Tku7oL0bnC8t//WoXX3r2tiWlluJK8JwlXqjI1PSUud7p5XQSSa96R7fJh7Fgb5vr553YC3dVXexE/V/oyDY9NnI3dBmioqmuA3YDTgI3qIDbnCsqiRdb/0Lw5vPkm3HSTXaLUuVKXzfDYp7HLoG4D3I+dQ/FQXqNyrkCowuTJdrtNG3jpJSsF3rNnvHE5V5eySRRrVXUVcCRwi6r+HmiV37Cci9/nn8OAAbDbbpVF/PbdFxo3jjcu5+paVpdCFZGjgaHAgHBfwRft9lFPrqZU7azqCy+0juobb/Qifq5+yyZRDAfOxMqMLxCRDsC/8xuWc/FAkh4mAAAVo0lEQVQ56ih48kkb1XTPPXZNE+fqsyoThap+ICLnANuKSCdgnqpem//QasdHPbnqiBbxGzAADjoITj3V6zM5B1n0UYjIXsA84F5gFPCxiPiBuCsZH3xgTUuJIn5Dh3qlV+eisvkq/C/QT1X3UNXdgUOAv+c3LOfy7+ef4aqroHt3mD8fNtkk7oicK0zZ9FE0VtVZiQlVnS0iBT/uwzuzXSbTp1sRvw8+gOOOg1tusUvmOud+LZtE8Y6I3AU8EKaH4EUBXZH79lvrx3r2WTj00Lijca6wZZMoTgfOAf4ICDAR+Ec+g8oF78x2yV57zYr4nXOOdVbPnQtNmsQdlXOFL2OiEJEdgW2Ap1T1hroJqea6dYOKChvOuGoVNCr4sz1cXfj+e/jjH2HkSOjUyTqq11/fk4Rz2cpUPfZPWPmOIcB/RCTVle4Kyty5lijA2ps7dow3Hhe/Z5+Fzp3tfIgLL7S+CS/i51z1ZDqiGALspKo/ikhLYCw2PLagNWoE48fHHYUrBIsWwcCBdhTx9NOw665xR+Rccco0PHalqv4IoKoVVcxbEFq08NFO9Z2qVXaFyiJ+06Z5knCuNjLt/LcWkSfD31PANpHpJzM87xci0kdE5ojIPBG5JMN8R4mIikhZdVfAuYTycjj8cDt5LlHEb599vIifc7WVqelpYNL0P6uzYBFpiF1r+0CgHJgqImOi52SE+Zpho6qmVGf5qfhIp/pp7Vq4+2646CJYvRpuvhn23DPuqJwrHZmumf1KLZfdA6sLtQBARB4G+gOzkub7H+AG4MJavp6rpwYOtD6I/fazhLH11nFH5FxpyWe/QytgUWS6nKTrWIhIN6CNqj6XaUEiMkJEponItIrEsCZXr61ebUcSYIni7rvh5Zc9STiXD/lMFJLiPv3lQZEGWB2pC6pakKqOVNUyVS1rmaHOgndm1w8zZ9rFhO6+26aPPx5OOcWqvzrnci/rRCEi1R19Xo5dbzuhNfB5ZLoZ0AUYLyKfAr2AMd6h7dJZuRKuuAJ22QU++8xrMzlXV7IpM95DRN4H5obpnUUkmxIeU4GOItIhFBEcDIxJPKiq36vq5qraXlXbA5OBw1V1Wk1WBKwz2zu0S9PUqVbl9eqr4dhjYfZsOPLIuKNyrn7I5ojiVuBQ4FsAVX0P2LeqJ6nqauBsYBwwG3hUVT8UkatF5PCah+zqoyVLYNkyGDsW7r8fNtss7oicqz+yKQrYQFU/k3UbgNdks3BVHYud0R29789p5t0nm2W6+uPVV62I3x/+YEX8Pv7Yy284F4dsjigWiUgPQEWkoYicC3yc57hcPbZ0qV2GdP/94a67rG8CPEk4F5dsEsUZwPlAW+ArrNP5jHwGVVM+6qn4PfOMFfEbNcoqvnoRP+fiV2XTk6p+jXVEO5dXCxfC0UfD9tvDmDFQ5uPfnCsIVSYKEbmbyPkPCao6Ii8R1UDiOhQVFX4NimKjCpMmwV57Qdu2dtJcr15en8m5QpJNZ/bLkdtNgCNY94zrWEQvUjQrFAVp2dLH1heThQvh9NPhhResNHzv3rD33nFH5ZxLlk3T0yPRaRF5APhP3iLK0ty5dhW7bbetTBAz/EreRWHtWrjzTrj4YjuiuPVWL+LnXCHL5ogiWQegXa4Dqa5Ep7VfpKj4HHmkdVofeKBdnrR9+7gjcs5lkk0fxRIq+ygaAN8Baa8t4Vwqq1dDgwb2N2gQ9O8Pw4Z5fSbnikHGRCF2lt3OwOJw11pV/VXHdhy8VEfxeO89GD7czo04/XQrweGcKx4Zz6MISeEpVV0T/goiSbjisGIFXH65DXMtL4cttog7IudcTWRzwt3bItI975G4kvL22zYy7dprYcgQK+I3YEDcUTnnaiJt05OIrBcK++0JnCoi84EfsetMqKp68nBp/fe/sHw5vPgiHHxw3NE452ojUx/F20B3oCB/B3qpjsLz0kvw4Ydw3nlwwAEwZ46X33CuFGRKFAKgqvPrKBZXpJYsgfPPh9GjYYcd4MwzLUF4knCuNGRKFC1F5Px0D6rqzXmIp0ozZ8I++3i5jkLx5JNw1ln2flx6Kfz5z54gnCs1mRJFQ6Apqa99HZvVq+2/l+uI38KFMHgwdOliFxTq1i3uiJxz+SDpRryKyDuF2GHdsGGZrllT46ululpShYkTrS4TWEG/nj396M65Qici01W1RjWZMw2PLagjCRe/zz6Dvn2t6W/CBLtvzz09SThX6jIliv3rLIpqaNgw7gjqn7Vr4Z//tI7qSZPgH/+wsuDOufohbR+Fqn5Xl4G4wjVgADz7rJ0Pcddd0C72kpDOubpUk+qxsVqzJu4I6odVq+zorUEDq8101FEwdKgX8XOuPsqmhIerZ955B3r0sGtGgCWKE07wJOFcfeWJwv1i+XI7F6JHD/jyS2jTJu6InHOFoOianrwzOz8mT4YTT4SPP7aS4DfeCJtsEndUzrlCUHSJwuXHjz9av8R//mN1mpxzLqHoEoV3ZufOiy9aEb8LLoD994ePPoLGjeOOyjlXaLyPoh769ltrZurbF+67D37+2e73JOGcS8UTRT2iCo8/Dp07w0MP2dXnpk71BOGcy6zomp5czS1cCMcdBzvtZNeO2HnnuCNyzhWDojui8FFP1aMKr75qt9u1g/HjbYSTJwnnXLaKLlG47H3yCRx0kHVUJ4r47b47rOfHkc65aii6ROGjnqq2Zg38/e92nYgpU+COO7yIn3Ou5vy3ZQnq3x+efx769bMyHH6GtXOuNjxRlIhoEb+hQ60+03HHeX0m51zt5bXpSUT6iMgcEZknIpekePx8EZklIjNF5BURqbKAtXdm/9q0aVBWZk1MAIMGwZAhniScc7mRt0QhIg2B24C+QGfgWBHpnDTbDKBMVXcCHgduyFc8pWj5crj4YrsUaUWFXyfCOZcf+Tyi6AHMU9UFqvoz8DDQPzqDqr6mqj+FyclA66oW6p3Z5q23bIjrDTdYEb9Zs+DQQ+OOyjlXivLZR9EKWBSZLgd6Zpj/ZOCFVA+IyAhghN3unqv4itry5XaJ0pdftuGvzjmXL/lMFKlayDXljCLHA2VA71SPq+pIYCRAw4ZlKZdRH4wda0X8LroI9tsPZs+GRo3ijso5V+ry2fRUDkQHZrYGPk+eSUQOAC4DDlfVlXmMp2h98w0cfzwccgg8+GBlET9PEs65upDPRDEV6CgiHUSkMTAYGBOdQUS6AXdhSeLrbBZan0Y9qcLDD8P228Ojj8IVV8Dbb3sRP+dc3cpb05OqrhaRs4FxQENglKp+KCJXA9NUdQzwN6Ap8JjYWM6Fqnp4vmIqNgsXWjnwnXeGe++FHXeMOyLnXH0kqsXV5N+wYZmuWTMt7jDyRhVeeaXyKnOTJ8Ouu9avIynnXO6JyHRVLavJc4uu1lMpmz/fRjAdeGBlEb9evTxJOOfi5YmiAKxZAzffbE1L06fDXXd5ET/nXOEoulpPpfjr+rDD4IUX7IS5O+6A1lWeduicc3Wn6BJFqfj5Z7suRIMGMGyYFfIbPNjrMznnCk/RNT2VQgmPt9+GXXaB22+36WOOsWqvniScc4Wo6BJFMfvpJ7jgAthtN1iyBLbZJu6InHOuat70VEcmTbJzIhYsgNNOg+uvh+bN447KOeeq5omijiQuLPTaa7DPPnFH45xz2Su6RFFMo56efdYK9/3xj7DvvlYKfL2i2+LOufrO+yjyoKLCLkN6+OHw739XFvHzJOGcK0ZFlygKedSTKjz0kBXxe/xxuPpqmDLFi/g554qb/8bNoYUL4aSToFs3K+K3ww5xR+Scc7VXdEcUhWbtWhg3zm63awevvw5vvOFJwjlXOoouURRSZ/bcuXaluT59YOJEu69Hj8KK0TnnaqvoEkUhWL0a/vY32GknePdda2byIn7OuVJVdH0UhdCZfeih1tzUv7+V4dhqq7gjcs65/Cm6RBGXlSvtGtUNGsApp8Dw4XD00V6fyTlX+rzpKQuTJ0P37nDbbTZ91FFWyM+ThHOuPvBEkcGPP8J558Huu8MPP0DHjnFH5Jxzda/omp7qakTR669bEb9PPoEzz4S//hU23rhuXts55wpJ0SWKurJ6tfVJTJgAe+8ddzTOORefoksU+Rz19PTTVsTv0kutiN+HH3p9Juec8z4K4KuvrHP6iCOsRpMX8XPOuUr1OlGowgMPQOfO8MwzcO21NsLJi/g551ylovvNnMvO7IUL7ZyIsjI7u7pTp9wt2znnSkW9O6JYuxZeeMFut2tnBfwmTvQk4Zxz6RRdoqhNZ/bHH9tlSPv1s9FMYEcTXsTPOefSK7pEUROrV8P111sRv/ffh3/9y4e8Oudctoquj6ImDjkEXnoJjjzSynBssUXcETnnXPEQVY07hmpp2LBM16yZVuV8K1bYCXMNG8ITT9h9AwfmOTjnnCtQIjJdVctq8tyia3rKpj/hjTega9fKIn4DB3qScM65miq6RJHJsmVwzjl2EaEVK2D77eOOyDnnil/R9VGkG/U0YYIV8Vu4EM4+G/7yF2jatG5jc865UlR0iSKTDTe0qq977BF3JM45VzqKOlE8+SR89BH86U/Qu7cNffVzIpxzLrfy2kchIn1EZI6IzBORS1I8vr6IPBIenyIi7ataZsOG8OWXdpW5gQPhqacqi/h5knDOudzLW6IQkYbAbUBfoDNwrIh0TprtZGCJqm4L/C9wfVXLXbvWOqmfe84uJvTmm17Ezznn8imfRxQ9gHmqukBVfwYeBvonzdMfuC/cfhzYXyTzlajXrIEuXeC99+CSS+xcCeecc/mTzz6KVsCiyHQ50DPdPKq6WkS+BzYDvonOJCIjgBFhcuWkSfKBF/EDYHOStlU95tuikm+LSr4tKv2upk/MZ6JIdWSQfBp4NvOgqiOBkQAiMq2mZxeWGt8WlXxbVPJtUcm3RSURqbqkRRr5bHoqB9pEplsDn6ebR0TWA5oD3+UxJuecc9WUz0QxFegoIh1EpDEwGBiTNM8Y4MRw+yjgVS224lPOOVfi8tb0FPoczgbGAQ2BUar6oYhcDUxT1THAvcADIjIPO5IYnMWiR+Yr5iLk26KSb4tKvi0q+baoVONtUXTVY51zztWtkioK6JxzLvc8UTjnnMuoYBNFPsp/FKsstsX5IjJLRGaKyCsi0i6OOOtCVdsiMt9RIqIiUrJDI7PZFiJyTPhsfCgiD9V1jHUli+9IWxF5TURmhO9JvzjizDcRGSUiX4vIB2keFxG5NWynmSLSPasFq2rB/WGd3/OBrYHGwHtA56R5zgTuDLcHA4/EHXeM22JfYMNw+4z6vC3CfM2AicBkoCzuuGP8XHQEZgCbhOnfxB13jNtiJHBGuN0Z+DTuuPO0LfYGugMfpHm8H/ACdg5bL2BKNsst1COKvJT/KFJVbgtVfU1VfwqTk7FzVkpRNp8LgP8BbgBW1GVwdSybbXEqcJuqLgFQ1a/rOMa6ks22UGDjcLs5vz6nqySo6kQyn4vWH7hfzWSghYhsWdVyCzVRpCr/0SrdPKq6GkiU/yg12WyLqJOxXwylqMptISLdgDaq+lxdBhaDbD4X2wHbicgbIjJZRPrUWXR1K5ttcSVwvIiUA2OB39dNaAWnuvsToHCvR5Gz8h8lIOv1FJHjgTKgd14jik/GbSEiDbAqxMPqKqAYZfO5WA9rftoHO8p8XUS6qOrSPMdW17LZFscCo1X1JhHZDTt/q4uqrs1/eAWlRvvNQj2i8PIflbLZFojIAcBlwOGqurKOYqtrVW2LZkAXYLyIfIq1wY4p0Q7tbL8jz6jqKlX9BJiDJY5Sk822OBl4FEBV3wKaYAUD65us9ifJCjVRePmPSlVui9DccheWJEq1HRqq2Baq+r2qbq6q7VW1PdZfc7iq1rgYWgHL5jvyNDbQARHZHGuKWlCnUdaNbLbFQmB/ABHZHksUFXUaZWEYA5wQRj/1Ar5X1S+qelJBNj1p/sp/FJ0st8XfgKbAY6E/f6GqHh5b0HmS5baoF7LcFuOAg0RkFrAGuEhVv40v6vzIcltcANwtIudhTS3DSvGHpYj8G2tq3Dz0x1wBNAJQ1Tux/pl+wDzgJ+CkrJZbgtvKOedcDhVq05NzzrkC4YnCOedcRp4onHPOZeSJwjnnXEaeKJxzzmXkicIVHBFZIyLvRv7aZ5i3fbpKmdV8zfGh+uh7oeTF72qwjNNF5IRwe5iIbBV57B4R6ZzjOKeKSNcsnnOuiGxY29d29ZcnCleIlqtq18jfp3X0ukNUdWes2OTfqvtkVb1TVe8Pk8OArSKPnaKqs3ISZWWct5NdnOcCnihcjXmicEUhHDm8LiLvhL/dU8yzg4i8HY5CZopIx3D/8ZH77xKRhlW83ERg2/Dc/cM1DN4Ptf7XD/dfJ5XXALkx3HeliFwoIkdhNbceDK+5QTgSKBORM0TkhkjMw0TkHzWM8y0iBd1E5A4RmSZ27Ymrwn3nYAnrNRF5Ldx3kIi8FbbjYyLStIrXcfWcJwpXiDaINDs9Fe77GjhQVbsDg4BbUzzvdODvqtoV21GXh3INg4A9wv1rgCFVvP5hwPsi0gQYDQxS1R2xSgZniMimwBHADqq6E3BN9Mmq+jgwDfvl31VVl0cefhw4MjI9CHikhnH2wcp0JFymqmXATkBvEdlJVW/Favnsq6r7hlIelwMHhG05DTi/itdx9VxBlvBw9d7ysLOMagT8M7TJr8HqFiV7C7hMRFoDT6rqXBHZH9gFmBrKm2yAJZ1UHhSR5cCnWBnq3wGfqOrH4fH7gLOAf2LXurhHRJ4Hsi5prqoVIrIg1NmZG17jjbDc6sS5EVauInqFsmNEZAT2vd4Su0DPzKTn9gr3vxFepzG23ZxLyxOFKxbnAV8BO2NHwr+6KJGqPiQiU4BDgHEicgpWVvk+Vb00i9cYEi0gKCIpr28Sagv1wIrMDQbOBvarxro8AhwDfAQ8paoqttfOOk7sKm7XAbcBR4pIB+BCYFdVXSIio7HCd8kE+I+qHluNeF09501Prlg0B74I1w8Yiv2aXoeIbA0sCM0tY7AmmFeAo0TkN2GeTSX7a4p/BLQXkW3D9FBgQmjTb66qY7GO4lQjj37Ayp6n8iQwALtGwiPhvmrFqaqrsCakXqHZamPgR+B7Efkt0DdNLJOBPRLrJCIbikiqozPnfuGJwhWL24ETRWQy1uz0Y4p5BgEfiMi7QCfsko+zsB3qSyIyE/gP1ixTJVVdgVXXfExE3gfWAndiO93nwvImYEc7yUYDdyY6s5OWuwSYBbRT1bfDfdWOM/R93ARcqKrvYdfH/hAYhTVnJYwEXhCR11S1AhuR9e/wOpOxbeVcWl491jnnXEZ+ROGccy4jTxTOOecy8kThnHMuI08UzjnnMvJE4ZxzLiNPFM455zLyROGccy6j/wc4kFM6e+zJqgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the Area Under the Curve is:  0.8582969635063604\n"
     ]
    }
   ],
   "source": [
    "plotROC(aggClassEst.T, labelMat)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
